Field of the Invention
The present invention relates to image processing apparatus and method which generate a new face image based on an input face image.
Description of the Related Art
Many kinds of techniques of extracting useful information by highly processing an image of an object included in photographed and imaged image data. Among them, there is the technique called face recognition of recognizing a human face image and then determining the attribute of the face. As examples of the face recognition, there are sex estimation of estimating a sex of a person from his/her face, age estimation of estimating an age of a person from his/her face, face discrimination of determining whether or not two face images correspond to a same person, and the like.
In the face discrimination, an image variation due to different photographing conditions for the two face images to be compared seriously influences accuracy of the face discrimination. Here, the photographing conditions causing the image variation include an illumination condition, a look, hiddenness, a secular change, a face direction, and the like. The photographing condition will be described by taking the face direction as an example. Here, the face direction indicates from which direction the face should be photographed. Generally, the face has a complicated shape. Therefore, the appearance of the photographed image remarkably and quickly changes when the front face portion in the photographed image changes according to the face direction. Particularly, in the face discrimination, it has been known that a change of appearance of the face image due to a different face direction of a same person is larger than a change of the appearance of the face image due to a different person, so that the former change highly prevents person discrimination (see “Face Recognition Across Pose—Review” X. Zhang, Y Gao: 2009).
In the face discrimination, there are roughly two methods of reducing variation influence. One is the method of deleting the information of a variation and remaining only the information of an original (natural) face. For example, as a method to be used in regard to a face-direction variation, the image conversion as described in “Towards Pose Robust Face Recognition”, D. Yi, Z. Lei, S. Z. Li: 2013 is representative. In this method, only the areas which are hardly influenced by the face-direction variation are compared to delete the face-direction variation. Therefore, since the areas of the face to be matched are reduced, there is a problem that the information of individual differences is deleted.
The other is the method of generating a new face image by estimating, from one face image, an appearance viewed on the photographing condition of the other face image, as described in “An Associate Predict Model for Face Recognition”, Q. Yin, X. Tang, J. Sun: 2011 (hereinafter, called the literature by Yin). By this method, it is possible to discriminate the two faces on the premise that the respective photographing conditions are the same (i.e., there is no variation). In the literature by Yin, the process is performed for each of the parts such as the eye, the nose and the like of the input face. For example, with respect to the eye, the instance (or case example) images of the eyes of many persons and the instance images obtained by photographing the eye of the same person respectively on the different photographing conditions have been previously associated and held. In the process, from among the held instance images of the eyes, one image which is most similar to the input eye image is selected. Then, the instance image on another photographing condition corresponding to the selected instance image of the eye is selected. By performing the above process to all the parts such as the eye, the nose and the like, it is possible to generate the image which can be seen as if the input face is photographed on another photographing condition.
In the literature by Yin, if the instance images of the persons to be previously held can cover the universe, it is possible to adequately estimate an appearance to be obtained on another photographing condition even in a case where any kind of face image is input. However, the number of the persons capable of being really held is finite. As a result, when the held images do not contain a person sufficiently similar to the input face, an image dissimilar to the target person oneself is generated.
Thus, the present invention aims to generate an image which is similar to the input face even in the case where the instance images do not contain the person sufficiently similar to the input face.